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ABSTRACT 

Face recognition (FR) is a challenging task in biometrics due 
to various illuminations, poses, and possible noises. In this 
paper, we propose to apply a novel neutrosophic set (NS)- 
based preprocesssing method to simultaneously remove noise 
and enhance facial features in original face images. We then 
employ the Tan and Triggs (TT) discriminant method, which 
applies kernel fisher linear discriminant analysis (KFDA) on 
the linear ternary pattern (LTP) features, on the NS -based pre- 
processed images to further improve FR accuracy. Our exper- 
iments on two databases (ORL and FEI) show that the NS- 
based preprocessing method is more effective than other pre- 
processing methods to improve FR accuracy of discriminative 
methods. It can also be integrated with other preprocessing 
methods to further improve FR accuracy. 

Index Terms — Face recognition, neutrosophic set, kernel 
fisher linear discriminant analysis, linear ternary pattern 

1. INTRODUCTION 

Face recognition (FR) is a good compromise between relia- 
bility and social acceptance and balances security and privacy 
well. It has been integrated in any application that requires 
security and access control [1]. However, illumination vari- 
ations, pose changes, facial expressions, age variations, and 
occlusions significantly affect the performance of a FR sys- 
tem [1]. Thus, various preprocessing methods have been in- 
corporated into the FR methods to achieve better FR accuracy. 

Linear normalization (LN) and histogram equalization 
(HE) are two popular preprocessing methods for FR [2, 3]. 
Recently, Tan and Triggs [4] proposed a preprocessor (PP), 
which uses a chain of operations such as Gamma correc- 
tion, Gaussian filtering, masking, and contrast equalization, 
to eliminate most of changing illumination effects. They also 
applied kernel fisher linear discriminant analysis (KFDA) on 
local ternary patterns (LTP) of the preprocessed image to 
achieve impressive FR accuracy. A comparative study of il- 
lumination preprocessing methods on FR [5] pointed out that 
HE, PP, gamma intensity correction (GIC), and logarithmic 
total variation (LTV) improve the FR accuracy. 



On the other hand, the neutrosophic set (NS), as an ex- 
tended version of the fuzzy set, has been recently used in im- 
age segmentation [6, 7] . Researchers use the notations T, /, 
and F to represent three neutrosophic components, e.g., truth, 
indeterminacy, and falsity degrees, respectively. So, a pixel in 
the NS domain is represented as P{T, /, F}, which means 
the pixel is t% true, i% indeterminate and f% false, where 
£, i, and / vary in T, /, and F, respectively. Zhang et al. 
[6] proposed a ^-function to map the intensity of each pixel 
to a value for T in [0,1], and then F = 1 — T. Sengur and 
Guo [7] used another transformation formula together with 
a-mean and /^-enhancement operations to map the intensity 
of each pixel to a value in {T, /, F}. 

In this paper, we propose to apply a novel NS -based pre- 
processsing method for FR. Specifically, we first use entropies 
of the indeterminate subset and the candidate preprocessed 
face image to find appropriate ranges for three NS -based pa- 
rameters. We then apply the NS operations on each im- 
age to simultaneously remove noise and enhance facial fea- 
tures. Finally, we uniquely combine this NS -based prepro- 
cessing method with the discriminative method to achieve 
better FR accuracy. To this end, we first apply the popu- 
lar KFDA on preprocessed images to evaluate FR accuracy 
on two databases. We then use one state-of-the-art discrim- 
inant technique, the Tan and Triggs (TT) method, as the 
core to evaluate the effect of different preprocessing meth- 
ods including the NS-based method, HE, LN, and PP on 
FR accuracy. Our extensive experiments on ORL and FEI 
databases demonstrate the following: 1) The proposed NS- 
based preprocessing method achieves the best FR accuracy 
when one preprocessing method is combined with the dis- 
criminant method. 2) The proposed NS-based method can be 
integrated with HE, LN, and PP to further improve FR accu- 
racy when paired with the discriminant method. The rest of 
this paper is organized as follows: Section 2 presents our pro- 
posed NS operations for preprocessing face images. Section 
3 shows the experimental results of applying the TT method 
on different preprocessed images obtained by one preprocess- 
ing method (NS, LN, HE, or PP) or combined prepocessing 
method (LN+NS, HE+NS, or PP+NS). Section 4 draws the 
conclusion and presents the directions of future work. 




2. NEUTROSOPHIC SET-BASED PREPROCESSING 

NS is a new branch of philosophy to study the origin, na- 
ture, and scope of neutralities, as well as their interactions 
with different ideational spectra [ 8 ]. Each event has three de- 
grees including truth, falsity and indeterminacy. We propose 
a novel preprocessing step for face images using NS-based 
operations. This step effectively removes noise and enhances 
facial features at the same time. Following NS-based segmen- 
tation techniques [ 6 , 7], we transform the original image into 
the NS domain, which maps the gray-level intensity of each 
pixel to a value in {T, /, F}. We then apply the a-mean and 
/3-enhancement operations on {T, /, F} to make the distribu- 
tion of the elements in I more uniform and the elements in T 
and F more distinct. Finally, we use a 7 -decomposition oper- 
ation to decompose T into the final preprocessed image. We 
use the entropy to empirically determine a , /3 and 7 values to 
achieve good preprocessing results for FR. 



where T a is the local mean value of T, F a is the local mean 
value of F, S T (i,j) = abs(T(i,j) - T(i,j)), 5 Tmin and 
S Tmax are respectively the minimum and the maximum of St 
at all positions in an image. 

Next, we employ a ^-enhancement operation to make the 
elements in T and F more distinct. It is defined as follows: 
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2 . 1 . a-mean, /3-enhancement, and 7 -decomposition oper- 
ations 



We transform a pixel P(i,j) with the intensity value of 
g(i,j) in the image domain into the NS domain, PNs(h j ) = 
{T(i,j),I(i,j),F(i,j)}. These values are defined as follow: 
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where g(i, j) is the pixel intensity at position (i,j),g(i,j) is 
the local mean value of g(i,j),S(i,j) = abs(g(i,j)-g(i,j )), 
Smin and Smax are respectively the minimum and the maxi- 
mum of local mean values at all positions in an image, and 
S m i n and Smax are respectively the minimum and the maxi- 
mum of 5 at all positions in an image. Here, we use the 3 x 3 
square window to compute the local mean values. 

The value of /(?’, j) measures the indeterminacy of ele- 
ment P N s(hj). We first employ an a-mean operation to in- 
crease the entropy of the indeterminate subset I and make 
the distribution of the elements in / more uniform. To this 
end, we make T and F correlate with I. In other words, the 
changes in T and F influence both the distribution of the el- 
ements and the entropy of I. This operation, smooths certain 
values in T and F based on the choice of ex. In addition, it 
increases the entropy of I. It is defined as follows: 

PNs(a) = P(T(a), 1(a), F (a)) (4) 
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where S' T (i,j ) = abs(T'(i,j) - T'(i,j)), and S' Tmin and 

S Tmax are respectively the minimum and the maximum of S' T 
at all positions in an image. 

Finally, we employ the 7 -decomposition operation to de- 
compose T into the final preprocessed image X. This opera- 
tion considers the effect of the indeterminancy I to recompute 

the value for the true subset T. It is defined as follows: 

_/ T(i,j) I(i,j)< 7 

T y (i,j) I(i,j)> 7 



X(i,j) = 



(14) 



where is the local mean value of the T set. 



2.2. Determination of a , /3, and 7 values 

We use the entropy of the indeterminate subset I to find the 
range of a and /3, and use the entropy of the candidate prepro- 
cessed image X to find the range of 7 . The entropy is chosen 
because its value can be interpreted as the quantity of infor- 
mation, i.e., the greater the entropy is, the more information 
the system includes [ 6 ]. The entropy of I is computed by: 

max{I } 

Em = - ^ pi log (pi) ; pi = Ni/N. (15) 

i=min{I } 

where Ni is the frequency of the rounded integer value i in I 
and N = We can similarly compute the entropy 

of X , i.e., Enx- 

To find a suitable range for a , /3, and 7 , for each com- 
bined a and /3 pair ranging from 0 to 1 with the stepsize of 
0.05, we perform the a-mean and /3-enhancement operations 
and find the (a, (3) pair with the highest entropy. Based on the 
results from the optimal (a, /3) pair, for each 7 value rang- 
ing from 0 to 1 with the stepsize of 0 . 01 , we perform the 7 - 
decomposition and find the 7 with the highest entropy. Fig. 
1 (a) shows Eni with regard to all experimented a and /3 val- 
ues for a face image. Fig. 1(b) shows Enx with regard to all 
experimented 7 values for the same image. Enj obtained its 
highest value when a and /3 are close to zero and almost less 
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Fig. 2. Sample images for one person and their preprocessed 

images from ORL and FEI databases 




Fig. 1. An entropy plot for a face image from ORL database. 
(Plots for other faces are similar) 



than 0.25, for almost 90% of face images from two databases. 
Similarly, Enx obtained its highest value when 7 value is 
close to zero but greater than both a and f3 values. So, we 
consider a=/3 and 7 = a + (3 in our system. The a value for 
each face is further computed by a linear scaling technique as: 



& — Q-min + 
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where En m ax = —log2j^, h and w are the height and width 
of an image, En m i n = 0 , a min =0 and a max = 0.25. 



2.3. Neutrosophic set-based preprocessing step 

We now use the cFs maximum value to quickly compute the 
optimal a value for each face image using Eq. (16). The al- 
gorithmic view of the proposed NS-based preprocessing step 
is summarized as follows: 

1. Transform the original face image into the NS domain, 

PNs(iJ) = {T(i,j),I(i,j),F(i,j)}, using Eqs 1-3. 

2. Compute the entropy Eni{old} of the subset /. 

3. Compute a value using Eq. (16). 

4. Assign a to /? and assign a+/3 to 7 . 

5. Sequentially perform the a-mean and the /^-enhancement 

operations on Pns( i , j) • 

6 . Compute the entropy Eni{new} of I. 

7. If (graj{n |7{~ofJ} J {° id }) < 8° t0 ste P 8 ’ else assi § n 

Enj{new} to Enj{old } and go to step 5. 

8 . Perform the 7 -decomposition operation to compute X. 

Our experiments on all face images show the convergence 
condidtion in step 7 is satisified after 3-4 iterations in most 
cases. As a result, this preprocessing step is fast after the 
offline process to learn the ranges for a, /3, and 7 values. 

3. EXPERIMENTAL RESULTS 

To evaluate the effectiveness of the proposed NS-based pre- 
processing method, we compare FR accuracy on two face 
databases (ORL and FEI) by combining different preprocess- 
ing methods with the discriminative method. To this end, we 
first apply the KFDA on the preprocessed images to evalu- 
ate its FR accuracy compared with FR accuaracy obtained 
by KFDA alone. We then use one state-of-the-art discrim- 
inant technique, the TT method, as the core to evaluate the 
effect of different preprocessing methods including NS, HE, 
LN, PP, HE+NS, LN+NS, and PP+NS on FR accuracy. For 



the kernels in KFDA and TT, we use the Gaussian kernel 
eX p(- dlst ( x X)/ 2 o- ), w h ere x and Y are vectors in the input 
space, dist(X , Y) is the % 2 histogram distance, and a = 10 5 
[4]. 

We use ORL and FEI databases, which have at least 
10 images per subject, to evaluate the FR performance of 
our preprocessing method since ORL database is one of the 
most popular face databases and FEI database is a color face 
database. We randomly select 4, 5, and 6 images per person 
for training and the remaining images for testing. To report a 
more accurate performance of each method under each train- 
ing, we average FR rates over 20 runs with each run being 
performed on a random partition of the databases. 

3.1. Recognition results on ORL database 

The ORL database contains 400 grayscale images (92x112 
pixels) of 40 individuals. They are taken at different times and 
slightly varying lighting conditions. They also have different 
facial expressions (open/closed eyes, smiling/non- smiling) 
and facial details (glasses/no-glasses). Fig. 2(a) shows a sam- 
ple individual from this database and the preprocessed images 
at the top and bottom rows, respectively. 

To speed up the processing time, we normalize all ORL 
face images to the size of 23x28 pixels. Table 1 shows the 
average FR rates obtained by combining different preprocess- 
ing methods and the discriminant method (e.g., KFDA or TT 
method) using a different number of training images. This ta- 
ble clearly shows that our NS-based preprocessing followed 
by KFDA improves KFDA by 0.93%, 0.89%, and 0.57% 
in terms of FR accuracy using 4, 5, and 6 training images 
per person, respectively. When combining one preprocess- 
ing method with the TT method, our NS-based preprocessing 
method achieves the best FR accuracy of 96.29%, 97.45%, 
and 98.31% using 4, 5, and 6 training images, respectively. 
When individually combining PP, HE, and LN with our NS- 
based preprocessing method, the combined preprocessing 
method achieves better FR accuracy than their corresponding 
preprocessing counterparts (PP, HE, and LN). Specifically, 
LN followed by the NS-based preprocessing achieves the best 
FR accuracy of 97.33%, 98.40%, and 98.87% using 4, 5, and 
6 training images, respectively. Correspondingly, it improves 
the TT’s FR accuracy (the 4th row) by 3.05%, 2.55%, and 
1.28%. It clearly shows the our NS-based preprocessing is 
the best one. 

3.2. Recognition results on FEI database 

The FEI face database has 14 images (640x480 pixels) for 
each of 200 individuals. They are color images with a white 



Table 1. Comparison of FR accuracy on ORL database. 



Compared methods 


Number of training samples | 


4 


5 


6 


KFDA 


93.83% 


95.50% 


96.01% 


NeutroKFDA 


94.70% 


96.35% 


96.56% 


N eutro+LTP+KFDA 


96.29% 


97.45% 


98.31% 


TT (PP+LTP+KFDA) 


94.45% 


95.95% 


97.62% 


HE+LTP+KFDA 


94.83% 


96.65% 


97.75% 


LN+LTP+KFDA 


94.87% 


96.50% 


97.56% 


PP+N eturo+LTP+KFD A 


95.70% 


96.80% 


97.87% 


HE+Neutro+LTP+KFDA 


96.79% 


98.00% 


98.68% 


LN+Neutro+LTP+KFDA 


97.33% 


98.40% 


98.87% 



Table 2. Comparison of FR accuracy on FEI database. 



Compared methods 


Number of training samples | 


4 


5 


6 


KFDA 


81.95% 


86.00% 


87.18% 


NeutroKFDA 


87.05% 


91.16% 


93.18% 


Neutro+LTP+KFDA 


90.60% 


93.33% 


95.00% 


TT (PP+LTP+KFDA) 


90.55% 


91.15% 


94.50% 


HE+LTP+KFDA 


88.50% 


91.22% 


93.50% 


LN+LTP+KFDA 


88.50% 


91.16% 


93.56% 


PP+N eturo+LTP+KFD A 


90.55% 


93.16% 


95.18% 


HE+Neutro+LTP+KFDA 


91.25% 


93.77% 


95.43% 


LN+Neutro+LTP+KFDA 


91.70% 


95.27% 


95.75% 



homogenous background in an upright frontal position with 
profile rotation of up to about 180 degrees. Subjects are be- 
tween 19 and 40 years old with distinct appearance, hairstyle, 
and adorns [9]. Fig. 2(b) shows a sample individual from this 
database at the top row and the corresponding preprocessed 
images at the bottom row. 

To speed up the processing time, we convert all FEI face 
images to grayscale images and normalize them to the size 
of 32 x24 pixels. We use a subset of this database that con- 
tains the first 40 individuals for our experiments. Table 2 
shows the average FR rates obtained by combining differ- 
ent preprocessing methods and the discriminant method (e.g., 
KFDA or TT method) using a different number of training 
images. This table clearly shows that our NS -based pre- 
processing followed by KFDA improves KFDA by 6.22%, 
6.0%, and 6 . 88 % in terms of FR accuracy using 4, 5, and 
6 training images per person, respectively. When combin- 
ing one preprocessing method with the TT method, our NS- 
based preprocessing method achieves the best FR accuracy 
of 90.60%, 93.33%, and 95.00% using 4, 5, and 6 training 
images, respectively. When individually combining PP, HE, 
and FN with our NS-based preprocessing method, the com- 
bined preprocessing method achieves better FR accuracy than 
their corresponding preprocessing counterparts (PP, HE, and 
FN). Specifically, FN followed by the NS-based preprocess- 
ing achieves the best FR accuracy of 91.70%, 95.27%, and 
95.75% using 4, 5, and 6 training images, respectively. Cor- 
respondingly, it improves the TT’s FR accuracy (the 4th row) 
by 1.27%, 4.52%, and 1.32%. It clearly shows the our NS- 
based preprocessing is the best one. 



4. CONCLUSIONS AND FUTURE WORK 

We propose an effective NS-based preprocessing method to 
remove noise and enhance facial features in face images. We 
then individually apply two discriminant methods, KFDA and 
TT, on preprocessed face images to achieve a better FR rate. 
Our experiments on two databases demonstrate the effective- 
ness of this NS-based preprocessing method to boost the per- 
formance of discriminative methods for the FR task. They 
also show the NS-based preprocessing can be integrated with 
other preprocessing methods to further improve FR accuracy. 

Our major contributions are: 1) Use entropy to find the 
best a , /3, and 7 values for all database images. 2) Find the 
relationship among a , /?, and 7 which is suitable for most 
face images. 3) Estimate the optimal a, /?, and 7 values for 
each face image. 4) Preprocess each face image using the op- 
timal a , /?, and 7 values by applying the entropy-based con- 
vergence condition. 5) Combine the NS-based preprocessing 
with KFDA or TT to achieve better performance. 

In the future, we will use clustering and statistical tech- 
niques to find a better relationship among a, /?, and 7 . We 
will also explore the possibility to employ NS-based prepro- 
cessing in color images and other applications. 
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